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The validity of various bootstrapping methods has been proved for the sam- 
ple mean of strongly mixing data. But in many applications, there appear 
nonlinear statistics of processes that are not strongly mixing. We investigate 
the nonoverlapping block bootstrap sequences which are near epoch depen- 
dent on strong mixing or absolutely regular processes. This includes ARMA 
and GARCH-processes as well as data from chaotic dynamical systems. We 
establish the strong consistency of the bootstrap distribution estimator not 
only for the sample mean, but also for [/-statistics, which include examples 
as Gini's mean difference or the x 2 -test statistic. 

keywords: strongly mixing sequences, near epoch dependence, [/-statistics, block boot- 
strap 
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1 Introduction 

1.1 Dependent Random Variables 

In many statistical applications the data does not come from an independent stochastic 
process. A standard assumption of weak dependence is given by the strong mixing 
condition: 
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Definition 1.1. Let (X n ) n&w be a stationary process. Then the strong mixing coefficient 
is given by 



a 



(k) = sup {\P(An B) - P(A)P(B)\ :A£J^,B£ T™ +k ,n£ W} 



where T l a is the a-field generated by r.v.'s X a , . . . , X[., and (X n ) n&¥j is called strongly 
mixing, if a(k) — > as k — > oo. 

For further information on strong mixing and a detailed description of other mixing 
conditions see Doukhan [1994J and Bradley [2007] . However, this class of weak dependent 
processes excludes examples like linear processes with innovations that do not have a 
density or data from dynamical systems. 

Example 1.2. Let (Z n ) n&v be independent r.v. 's with P [Z n = 1] = P [Z n = 0] = \ and 



oo - 



k=n 



Then X n= \ = 2X n [mod 1] and (X n ) n&v is not strong mixing, as 
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We will consider sequences which are near epoch dependent on strongly mixing or 
absolutely regular processes, as this class covers the example above and other interesting 
examples and data from other dynamical systems, which are deterministic except for the 
initial value. Near epoch dependent functionals of mixing processes have been studied 
for a long time, see for example Ibragimov [1962J or Billingsley [1968 . 

Definition 1.3. Let (X n ) n&K be a stationary process. 

1. The absolute regularity coefficient is given by 

(3{k) = suptfsupflPtA/J^) - P(A)\ : A e 

neIN 



and (X n ) ng]N is called absolutely regular, if (3(k) — > as k 



oo. 



2. We say that (X n ) n&K is near epoch dependent on a process (Z n ) n£ %, with approxi- 
mation constants (ai)i^, if 



E 



X l -E{X 1 \F l _ 



< 



1 = 0,1,2... 



where lim/^oo ai = and J- 1 ^ is the a-field generated by Z_\, . . . , Z\ 
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We consider L near epoch dependence, which is a weaker assumption than the more 
commonly used I? near epoch dependence. 

Example 1.4. Expanding Dynamical Systems: Let T : [0, 1] — > [0, 1] be a piecewise 
smooth and expanding map such that inf ze j 0j i] |T'(x)| > 1. Then there is a stationary 
process pT n ) ng]N such that X n+ i = T (X n ) which can be represented as a functional of 
an absolutely regular process (see Hofbauer, Keller 1198^ ). 

Example 1.5. GARCH: Let (Z n ) n ^ be a sequence of independent standard normal 
random variables and X n = a n Z n , where (cr n )new is a random sequence with <7 2 = 
ao + aiX^_i + OL2<rfi—\- Such GARCH(1,1) processes are used for modeling volatility 
clustering in financial data and are near epoch dependent with an exponential decay of 
the approximation constants, see Hansen fTUUJjj. 

Example 1.6. Volterra series: Many causal processes can be represented by a Taylor 
series. Assume that (Z n ) n ^ is a sequence of independent identically distributed and 
centered random variables with finite variance. We define 

oo oo 

X n = ^2 Y 9l(ui,...,ui)Z n _ Ul ...Z n n-ui. 

1=1 ui,...,ui=0 

Let gi(ui, . . . ,U[) = if Ui = uj for some then X n is near epoch dependent with 
approximation constants 

oo oo 

Yl 9 2 (ui,---,ui). 

1=1 u\,...,ui=k 

Volterra series can be understood as Taylor expansions for causual time series, so they 
include a broad class of models. For more details, see Rugh 11981}/ . 



a k <C 



\ 



1.2 {/-Statistics 

[/-statistics play an important role in nonparametric statistics because many estimators 
and test statistics can be written at least asymptotically as [/-statistics. Well-known 
examples include the sample variance, Gini's mean difference, and the x 2 goodness of fit 
test statistic. A more recent example is the Grassberger-Procaccia dimension estimator, 
[/-statistics can be described as generalized means, i.e. means of the values of a kernel 
function h (X^, . . . , Xj. ). For simplicity of notation, we concentrate on the case of 
bivariate [/-statistics: 

Definition 1.7. A U -statistic with a symmetric and measurable kernel h : R 2 — > R is 
defined as 

U n (h) = - 2 - Y h{Xi,Xi). 
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The key tool in the analysis of [/-statistics is the Hoeffding-decomposition [1948] of 
U n (h) into a so-called linear part and a degenerate part 

2 n 

U n (h) = + -Y,hi + u n (M 

i=l 

with 

6 :=Eh (X,Y), 
hi(x) := Eh(x,Y) - 6, 
h2{x,y) := h(x,y) - hi(x) - h x (y) - 6. 

for X, Y independent and with the same distribution as X\. We will need some more 
technical conditions on the kernel: 

Definition 1.8. Let {X n ) n ^ be a stationary process. 

1. A kernel has uniform r -moments, if there is a M > such that for all k G Mo 

E\h(X 1 ,X k )\ r < M and E \h(X, Y)\ r < M 
for X , Y independent and with the same distribution as X\. 

2. A kernel h is called F '-Lipschitz- continuous with constant L > 0, if 

E [\h(X,Y) -h{X',Y)\l {]x _ xim ] < Le 

for every e > 0, every pair (X,Y) with the same common distribution as (X\,Xk) 
for some k £ IN or independent with the same distribution as X\ and (X',Y) also 
with one of these common distributions. With 1a, we denote the indicator function 
of a set A. 

The Lipschitz-continuity condition is rather mild, some kernel with junps satisfy it, 
see Dehling, Wendler |2010a] . Furthermore, we have the following: 

Lemma 1.9. 1. Let h be a polynomial kernel of degree d, that is 

d d—i 

h(x,y) =J2J2 °y ( xlyj + xjyi ) ■ 

If ElXil^ 1 < oo, then h is P- Lipschitz- continuous. 

2. Let h be a P- Lipschitz- continuous kernel and / :E->Ra Lipschitz- continuous 
function. Then g o h is P- Lipschitz- continuous. 

Proof. 1. We can concentrate on an expression of the form g (x, y) = x l y 3 , i + j < d: 

E[\g{X,Y)-g(X , ,Y)\l { \ x _ x , ] < e} ] 
=E [| (X - X') {X 1 - 1 + X l - 2 X' + ... + XX H - 2 + X H - r ) Y'\ l { |x-X'|< £ }] 
<eE [| (X*- 1 + X' L - 2 X' + ...+ XX H - 2 + X' 1 - 1 ) Y j \] < eiE {X^ 
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2. This is obvious. 

□ 

Whereas the limit theory for partial sums of weakly dependent processes is very well 
developed, much less attention has been paid to nonlinear statistics like [/-statistics. 
The summands of U n Q12) can be correlated, if the random variables (A" n ) ng]N are de- 
pendent, so one has to establish generalized covariance inequalities to derive moment 
bounds for U n (h>z). Yoshihara [1976 J considered absolutely regular processes, Denker 
and Keller [1.986J functionals of absolutely regular processes, and Dehling and Wendler 
[2010a] strongly mixing sequences. 



1.3 Block Bootstrap 

In many statistical applications, for example in the determination of confidence bands, 
one faces the task to compute the distribution of a statistic T n = T n (Xi, . . . , X n ). This 
is a challenging problem if the random variables are dependent and the function T n is 
nonlinear. Therefore, block bootstrapping method are commonly used for nonparametric 
inference. There are different ways to resample blocks, for example the circular block 
bootstrap or the moving block bootstrap (for a detailed description of the different 
bootstrapping methods see Lahiri [2003]). For the circular block bootstrap, Shao and 
Yu |1993| have shown that under strong mixing the distribution of the block bootstrap 
version X* of the sample mean converges almost surely to the same distribution as the 
sample mean X n . Peligrad [1998] has proved asymptotic normality of X* under another 
set of conditions, which does not necessarily imply the central limit theorem for X n . 
Radulovic [1996] has established weak consistency under very weak conditions. 

Gongalves and White [2002] have proved the weak consistency under near epoch depen- 
dence. As far as we know, strong consistency has not been proved under such conditions, 
neither for the sample mean nor for [/-statistics. 

We consider the nonoverlapping block bootstrap, proposed by Carlstein [1986J, for the 
sample mean and for [/-statistics. Let (X n ) ngM be a sequence of r.v.'s. Let p 6 IN be the 
block length such that p = p(n) = o(n), p — > oo as n — > oo. We introduce the following 
blocks of indices and r. v.'s: 

L = (-X"(i-l)p+l3 • • • >-^ip) > 
Bi = {(i-l)p + l,...,ip}, i = l,...,k 

is the number of blocks. We consider a new sample X*, . . . , X| . 
ay choosing randomly and independently blocks k times with 

p((xi,...,x;) = i i ) = ± i = i,2,...,fc. 

As a bootstrap version of the sample mean we consider: 

kp 

Y~* \ v~* 



where k = k(n) - 
which is constructed 
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With P*, E* , Var* we denote the probability, expectation and variance conditionally on 
(X n ) nm . Note that 

kp 

& ^n,kp - J- / — ■ ^n,kp- 

P i=l 

The first aim of this paper is to prove the weak and the strong consistency of the 
nonoverlapping block bootstrap for sequences that are near epoch dependent on strongly 
mixing processes (Theorems l2.2l and l2,3p . as this class of weak dependent processes covers 
examples that do not satisfy the strong mixing conditions. 

Our second aim is to prove the weak and the strong consistency of the nonoverlapping 
block bootstrap for {/-statistics. Although the estimation of the distribution for U- 
statistics is even more complicated than for the sample mean, there is only very little 
literature on the bootstrap for {/-statistics. Bickel and Freedman [1981] proved the 
validity of the bootstrap for nondegenerate {/-statistcs of i.i.d. data, Arcones, Gine 
[1992] . Dehling, Mikosch [1994], and Leucht, Neumann [2009] for degenerate {/-statistics 
of i.i.d. data. Dehling and Wendler [2010a] have shown that the weak constistency of the 
circular block bootstrap for nondegenerate U -statistics of strongly mixing or absolutely 
regular sequences. 

2 Main Results 

In this section, and in what follows, we denote by X n the sample mean of the observations 
X\, . . . ,X n , by iV(0,cr 2 ) a Gaussian r.v. with mean zero and variance a 2 and by C a 
constant which may depend on several parameters and might have different values even 
in one chain of inequalities. First we will give theorems for the nonoverlapping block 
bootstrap and general stationary sequences which are analogues to the results of Peligrad 
[1998J, and Shao and Yu [1993] for the circular block bootstrap. 

2.1 Bootstrap for the sample mean 

We formulate theorems for the nonoverlapping block bootstrap. We will first show the 
weak constistency under strong mixing and near epoch dependence on a strongly mixing 
process. The first part of the following theorem (strong mixing) can be found in the 
book of Lahiri [2003] . 

Theorem 2.1. Let (X n ) n< -j^ be a stationary sequence r.v.'s with EX\ = //, for some 
5 > 0: .El-Xil < oo. Assume that l/p(n) + p{n)/n — >■ as n — >• oo and that one of 
the two following conditions holds 

1. (X n ) nem is strongly mixing with Ylk=i aT ^(^) < °°- 

2. (A" n ) ngW is near epoch dependent (with approximation constants (ai)i e ^) on a 
strongly mixing process process (Z n ) neZ and ^2 < kLi(o-l +s + ot 2 + s (k)) < oo. 
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Then a 2 = EX 2 + 2 Cov(Xi,Xi) < oo and in the case a 2 > 

Var*(yfcpX* kp ) — )■ a 2 in probability 



sup 



P* (^/kp~{X^ kp - X n , kp ) <x)-P {Vn~(X n -/i)<x) 



(1) 

in probability 
(2) 



Remark 2.1. This Theorem is also valid for the moving or the circular block bootstrap, 
we skip the introduction of these two bootstrap methods. This Theorem weakens the 
moment assumption of Gongalves and White fHUOttf (they considered not only sequences, 
but also triangular arrays.) 

The first part of the following theorem (strong mixing) is analogues to the results 
of Peligrad H)!)8 and Shao, Yu [1993J for the circular block bootstrap, showing strong 
consistency. 

Theorem 2.2. Let (X n ) nl - K be a stationary sequence r.v. 's with EX% = fj,, p(n) — > oo, 

p(n) < Cn e for some < e < 1 and (3) 

p(n) =p{2 1 ) for 2 l < n < 2 l+1 , 1 = 1,2,... (4) 
and assume that one of the two following conditions holds: 

1. Let be {E \ X 1 1 z+0 ) w < oo for some < 5 < oo. Assume that (X n ) n ^ is strongly 
mixing with a(k) = 0(k~ a ) for an a > 

2. Let be E\X\\ A+S < oo for some 5 > 0. Assume that (X n ) ngW is near epoch de- 
pendent (with approximation constants (ai) l& j^) on a stationary absolutely regular 

s s 

process {Z n ) n& and E^o^K^ + (P( k ))~) < °°- 
Then a 2 = EX\ + 2 Y,T=2 Covpfi, X;) < oo and in the case a 2 > 

V a r*(v^X: ikp )^a 2 a.s. (5) 
P* (y/kp~(X* >kp - X rhkp ) <x)-P {yfn~{X n - n) < x) -)■ a.s. (6) 



sup 

Remark 2.2. Line is a technical condition on the block lenght that is needed for the 
chaining techniques we will use. The conditions in the second part of this Theorem are 
the same as for the Central Limit Theorem in Borovkova et al. I2001f . 

Theorem 2.3. Let (X n ) n( - K be a stationary sequence of almost surely bounded r.v.'s. 
Assume that p(n) — > oo, 

p(n) = p(2 l ) for 2 l <n< 2 l+1 , 1 = 1,2,..., 

p 2 /n — > 0, and one the two following conditions holds 
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1. (X n ) n< --^ is strongly mixing, M5\) holds and 



P 2 (n)a(p(n)) 



n 

n=l 



oo 



^ < oo. (8) 



n 

n=l 



(X n ) ng ]N is near epoch dependent (with approximation constants (a;) ig]N j on a 
stationary absolutely regular process (Z n ) n&z with ^2^=0 ^ 2 ( a k + < °°- 



T/ien 

" L n,fcp 



Var^T^X^J^a 2 a. S . (9) 



/ kp(X* kp — X n ^ p ) — > N(0,a 2 ) in distribution (10) 
almost surely. 

Remark 2.3. The condition ^ implies X^nLi ~7T < 00 • ^ e mn reformulate the first 
part of Theorem \2.3\ under above condition on mixing coefficients instead of conditions 
([?P and claiming that there is a sequence (p(n)) that the statement of Proposition 
\3.12\ holds (although the Central Limit Theorem has not to hold), as it was done in 
Peligrad [1998]. Under the conditions of the second part of Theorem \2.3l the Central 
Limit Theorem is true for X , so that the statement of Proposition \3.12] is also valid. 

2.2 Bootstrap for U-Statistics 

To bootstrap a [/-statistic under dependence, one can apply the nonoverlapping block 
bootstrap and plug the observations X*, . . . ,X* in: 

y ^ ' l<i<j<pk 

2 Pk 2 
1 i=l 1 ^ ' l<i<j<pk 

Theorem 2.4. Let (X n ) n&K be a stationary process and h a P- Lips chitz- continuous 
kernel with uniform (2 + 5) -moments for a 5 > 0. Assume that l/p(n) +p(n)/n — > 
and one of the following two conditions holds: 

1. (X n ) ngW is strongly mixing, E |Xi| 7 < oo for a 7 > and a (n) = O (n~ a ) for a 

3 7 «5+<5+5 7 +2 
a > 2-yS 

2. (X n ) nl -j^ is near epoch dependent on an absolutely regular process with (5 (n) = 
O (n-l 3 ) for a (3 > ^ and a{n)=Q (n~ a ) for an a > 
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then as n —> oo 



Var* 



pkU* (h) - Var [yfnU n (/»)] 0, 



sup 



pfc(Z7*(/i)-S*[C/*(/t)]) <x -P[,/n(U n (h)-0)<x] -> 0. (12) 



in probability. 



Theorem 2.5. Zei (X n ) ngIN oe a stationary process and h a P-Lipschitz- continuous 
kernel with uniform (2 + 5)-moments for a 5 > and E \h\ (Xi)\ 4+S . Assume that ([3j], 
Q), and one o/ t/ie following two conditions holds: 



1. (X n ) nl -j^ is strongly mixing, E |^i| 7 < oo for a 7 > and a (n) = O (n a ) /< 



or a 



a > 



37<5+<5+57+2 
27<5 



2. {X n ) n p^ is near epoch dependent on an absolutely regular process with f3 (n) 
O (n-P) for a f3 > ana > a ( n ) = o (n _ °) /or an a > 



t/ien a.s. as n-> 00 



Var* 



pA:?7* (/i) - Var [Vnf7„ (/»)] -> 0, 



(13) 



sup 



P* 



pk(U*(h)-E*[U*(h)])<x -P[y/n(U n (h)-9)<x] -»• 0. (14) 



3 Preliminary results 

3.1 Central Limit Theorem, Moment and Maximum Inequalities for Partial 
Sums 

In this subsection we will give some known results which will be used in the next section 
in the proofs of the theorems. We set 

11 

i=i 

and assume w.l.o.g. that EXi = 0. 

Lemma 3.1 (Ibragimov [1962 J). Assume that one of the following conditions is satisfied: 

• Let (X n ) n£K be a stationary sequence of strongly mixing r.v.'s with EX\ = /x and 
[E \X\\ ) 2 + s < 00 for some < 5 < 00. Assume that 



^a 2 +« (k) < 00. 



k=i 
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Let (X n ) neW be a stationary sequence that is near epoch dependent on a strongly 
mixing such that 



. s 

2_^a2+s(k) < oo 
k=l 

oo 

Y,{e\Xo-e[x \z^,...,z 1 }\ 2 ^ 



k=l 



1+1 

2+5 



< OO 



VarS n 



a 2 . If in addition a 2 > 0, 



and (E\X\\ ) 2 + s < oo for some < 5 < oo. 

Then a 2 = VarX 1 + 2 Y%=2 Cov(Y, X) < oo and 
i/ien 

n 1 / 2 (X n —//)—>■ iV(0,(7 2 ) in distribution. 

Lemma 3.2 (Shao [1993J). Let {£ n ) n ^ be a strongly mixing sequence of r. v.'s with 
E£i = and (E 1/1,8 < for 1 < i < n and for some 1 < s < oo. Assume that 

a(i) < Coi~ e for some Cq > 1 and 9 > 0. 

TTien i/iere exists a constant K = K (Co, 0, s), suc/i i/iai /or any x > KV n n l l 2 logn 



P(max 



i=i 



7? s(e+i) r 
> x ) < Kn( — )^+^(log — 
x V n 



Lemma 3.3 (Shao and Yu |1993j ). Let (£ n ) ne ]N be a sequence of r. v. 's with E£i = 0. 
Assume that there is a constant C > such that for any n > 1 



Then 



almost surely. 



/ k+n \ 2 

sup £ V & ] < Cn. 



1 

lim — - — — V & = 

v ° 2=1 



Lemma 3.4 (Yokoyama [1982]). Let (X n ) n< -^ be a stationary strongly mixing sequence 

o/ r.v.'s with EX\ = fj, and [E\Xi\ ) 2 + s < oo for some < 5 < oo suppose that 
2 < s < 2 + 5 and 



(2+(5-s)/(2+<5) 



< OO. 



n=l 



Then there exists a constant C depending only on s, 5 and the mixing coefficients (a(n)) ng]N 
such that 



E 



< Cn 8 / 2 (S|Xi| 2+<5 )^. 
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Lemma 3.5 (Rio [1995], Peligrad [1998J). Let (X n ) ng]N be a strongly mixing sequence 
of r. v.'s with EX{ = and \Xi\ < C a.s. Then there is a universal constant K such 
that for every x > and n > 1 

P ^ max \Si\ > x^j < Kx~ 2 [^EX 2 + C 2 ■ n^a(i)| . 

Lemma 3.6 (Borovkova et al. [2001J ) . Let (X n ) neN be a stationary sequence which is 
near epoch dependent (with approximation constants {ak)ke¥l) on an absolutely regular 
process (Z n ) n£X with mixing coefficients (I3(k)) ke¥j . Suppose that one of the following 
two conditions holds 

1. EX = 0,E\X \ 4+S < oo, 

££Li k 2 {af~ s + (P(k))£s) < oo for some 5 > 0. 

2. Xq is bounded a.s.,EXo = 0, 

£r=i^ 2 K+w)<°o. 

Then a 2 = EXq + 2 X/fcLi EXqX^ < oo and in the case a 2 > we have 

= S n — > N(0,a 2 ) in distribution as n — > oo. 
n 



Lemma 3.7 (Borovkova et al. [2001] ). Let (A" n ) ng]N be a stationary sequence which is 
near epoch dependent (with approximation constants (ffljOfceW on an absolutely regular 
process {Z n ) n ^ with mixing coefficients {P(k)) k ^. Assume that one of the conditions 
of Lemma \3.6\ holds. Then there exists a constant C such that 

ESi < Cn 2 . 

Lemma 3.8 (Borovkova et al. [2001J). Let (X„) ngH be a stationary sequence which is 
near epoch dependent (with approximation constants (afc) fcgfJy ) on an absolutely regular 
process {Z n ) n£X with mixing coefficients (/3(A;)) fcg]N . Assume that EXq = and one of 
the following two conditions holds: 

1. Xq is bounded a. s. and J2'kLo( a k + P(k)) < oo, 

2. E\X G \ 2+S < oo and ET=o( a f* + (P( k ))^) < °°- 
Then there exists a constant C such that 

ES 2 n < Cn. 

Lemma 3.9 (Borovkova et al. |2UUlj ). Let (X n ) ng]N be a stationary sequence which is 
near epoch dependent (with approximation constants (ak) k&K ) on an absolutely regular 
process {Z n ) n ^ with mixing coefficients (P(k)) ke ^. 
Then 



11 



Bootstrap for Near Epoch Dependent Processes 



1. if \Xq\ < M a. s. for all non-negative integers i < j < k < I, we have 

\EiXiXjXkXt) - E{XiXj)E{X k Xi)\ < 

^4(/3([^]))^(S|Xo| 2+5 )^+8(a [ ^_ 2] )^(i?|X | 2+<5 )TTi^ .M 2 . 

2. if E \X$\ i+S < oo for all non-negative i < j < k < I, we have 

\EiXiXjXkXO - EiXiX^EiXnXi^ 

<mi^]))^(E\X \ 4+S )^ +8(a [! ^ ] )^(E\X \* +5 )£s. 

3.2 Moment Inequalities for U-Statistics 

To control the moments of the degenerate part of a {/-statistics, we need bounds for the 
covariance of h 2 - Recall that h 2 is defined as 

h 2 (x,y) := h(x,y) - hi(x) - hi(y) - 6. 

Lemma 3.10 (Dehling, Wendler [2010b]). Let (X n ) neJN be a stationary process and h a 
P-Lipschitz- continuous kernel with uniform (2 + 5)-moments for a 8 > 0. Let be r > 
such that one of the following three conditions holds: 

27<S 

1. (X n ) nl - K is strongly mixing, E |Xl| 7 < oo for a 7 > and X]fc=o ka~t s + s+h ~t+ 2 (k) = 
0(n T ). 

2. (X n ) n€JN is a near epoch dependent on an absolutely regular process and for Al = 
Then: 

n 

J2 \E[h 2 (X h ,X i2 )h 2 (X i3 ,X i4 )]\ = 0(n 2+T ). 

*li*2,«3i*4 = l 

If (X„) ng ]N is near epoch dependent, it is not clear that the same holds for {h\{X n )) n ^. 
The following Lemma is very similar to Proposition 2.11 of Borovkova et al. [2001j and 
gives an answer: 

Lemma 3.11. Let (X n ) n€ ^ be L 1 near epoch dependent on the process (Z n ) n ^z with 
approximation constants (a/)z e ]N and g satisfy the variation condition with constant L. 

1. If E\g(Xi)\ 1+s < 00, then (g(X n )) n& ^ is near epoch dependent on (Z n ) n6 ^ with 
approximation constants 

a' l = (L + 2W-\\ g (X 1 )\\ 1+s )af^. 

2. If g{X\) is bounded, then (g(X n )) nG ^ is near epoch dependent on (Z n ) n ^ with 
approximation constants 

a' l = {L + 2\\g{X 1 )\\ 00 )a}. 
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3.3 General Propositions for the Bootstrap of Stationary Sequences 

The proofs of Propositions 13.121 and [37T51 are mainly based on the methods developed in 
Peligrad [1998] and Shao, Yu [1993] . We will give full proofs for completeness. 

Proposition 3.12. Let {Xi,i > 1} be a stationary sequence of r.v'.s such that EX\ = \i 
and VarXi < oo. Assume that the following conditions hold 



Var n 1/2 (X n - p) -»• a 2 > 0, 

n l l 2 {X n — /u) — > N(0,o~ 2 ) in distribution, 

V l,2 {X n ,k P ~ M) a.s. , 

2 



-y 

kp ^— ' 

^ 8=1 



U'6-Bi 



i=i \jeBi 



vies* 



a.s. 







a.s. 



(15) 
(16) 
(17) 

(18) 
(19) 



for any e > 0. Then the following takes place as n — )• oo 

Var*(y^kp'xi kp )^a 2 

sup P* (y^(K k P ~ X n ,k P ) <x)-P (V^{X n -/*)<*) -+ 
xeM. v ' 7 

Proof. We note that 



a.s. 
a.s. 



kp(Xn,kp ~ ^n,kp) — —Jf E Z* n 



k 

where Z* n = SjeBjO^i ~ ^n,k P ), i = l,...,k are i.i.d. r.v.'s (conditionally on 
(X n ) ngIN ). By simple calculations we have 



= i E f ( E - ^) 2 - E ( E (*j - )+l^s p - P (x n 

P i=l \ j6Bi jGBi / p 

Conditions (USD, (fT7|) and (fig]) imply that a.s. as n — > oo 

E*Z* 2 n — > a 2 , and consequently Var* ^/kpX* kp 

For any e > 0, we have 



,fcp — 



(20) 



1 

£7 *^> 1 {Z|„>6fc} = IT E( E ~ X n,kp)) 2 ^ 

1 i=i jes 8 
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4 k 



^Tp E< E (*i " ^^ 2l {(EjeB i (- x i-A*)) 2 > £ 3 E } 

4 fc 
^ i=l 



4 

Now lines ([IT]) and CE} imply that 

E*Zif n lf( Z * 2 >efc | — >■ a. s. asn->oo (21) 



what means that Z* n , i = 1, 2 . . . satisfies the Lindeberg condition. Thus lines (j20j) and 
(|2ip imply the statement of the theorem. □ 



Proposition 3.13. Let (X n ) n£K be a stationary sequence ofr.v. 's. with EX\ = fi,Vaj:Xi < 
00. Assume that conditions \15\) . M6\) . M8\) and for each fixed x € R 

-f-Vul fl ^ tv , ! - P f— - u) < 2^ -> a.s. (22) 

/10W. T/ien i/ie statement of Proposition \3.12\ remains true. 
Proof. We define 

fc 

fe^^iEjV^^-M)^}- 

It is easy to see that the r.v.'s 



1 tp 

~j= E (-X?-M)^i = l,...,* 

V^i=(i-l)p+l J 

are i.i.d. with distribution function F n {x). We denote by Fn the distribution function 
of 

, k I i P \ 

Note that 

x 2 dp n (x) =i- E (E ( x i - ^)) 2 - s (E( x i - ^)) 2 + ^(£(*i - /^)) 2 
y v £ ( (E - ^)) 2 - *(£(** - ^)) 2 ) + ^ Var ^- 

1 i=l \ jdB, j=l j P 
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From the lines (|15p and (|18p we have 

/* W „' a.s.asn^oc. 

Now conditions (|16p and ()22[) imply 

F n (x) -»• iV(0,CT 2 ) a.s. 

The rest of the proof is the same as in the proof of Theorem 2.2 of Shao and Yu [1993J . □ 

The following proposition is analogue of Proposition 3.1 of Peligrad [1998J (In this 
proposition we assume that (x n ) rag]N is a fixed realization of (X n ) ngM ). 

Proposition 3.14. Let (x n ) n ^ be a bounded sequence of real numbers. For each n, 
let T n i, T n 2, ■ ■ ■ P n k be independent r.v. 's uniformly distributed on {1, 2, . . . , k} . Assume 
that condition A23\) and 

k 

V n = T Y^P* ~ ^n,kp) 2 -> 0- 2 > 



k 

i=l 



hold, where x pi = | Y,jeB z x j> x n, P k = ^ YliLi x i- Then 



k k 



Vkp( X n,kp - x n,kp) = Vkp{v Yj ILj 1 {Tn J =i} X Pi ~ X n,kp) ~> N(0, a 2 ) 



'k 

j=\ i=l 



in distribution. 
Proof. We have 



YsxX n f, p — ^ 2 Y^( X pi x n,kp) 



1=1 

and by line (|16p obtain 



k 

i=l 



Note that 

k 



k P( X n,kp ~ x n,kp) = V^Piz 1 {Tn J =i} X Pi ~ X n,kp) 



j=l i=l 

k k 



^(~k ^ ^ ^{T nj =i}( x pi ~ X n,kp)) = U "3> 
j=l i=l j=i 



15 



Bootstrap for Near Epoch Dependent Processes 



where 

k 



■Epi %n,kp) 



In our case Lindeberg condition holds if 



E max U„a — >• as n — )• oo. 
i<j<fc J 

Taking into account that r. v.'s are bounded we have 
and 

|ty»-o(£)-o(£) 

Now the condition (j23j) implies the statement of the proposition. □ 

Proposition 3.15. Let (X n ) n ^ be a stationary sequence of bounded almost surely r.v. 's 
with EX\ = fj,. Assume that conditions t!5\). |-?7| ) U8\) and following conditions hold 

p 2 

— ->• as n -»• oo. (23) 

n 

T/zen Zme (0) remains true and 

\/kp(X* kp — X n ^ p ) — > N(0,a 2 ) in distribution. (24) 
Proof. The proof is based on the previous proposition, so we have to show that 

k 

V n = j- y2(X pi - X n ^ kp f -»■ a 2 a.s. as n oo. 
W.l.o.g. assume that fi = EX\ = 0. Set S p i = YljeBt Xj. Then 

k k q2 

V - ?-X^ Y 2 - r,Y 2 - Pi nY 2 

n L, Pi i JjX n,kp " , ^ 2 P^n,kp 

1=1 8=1 1 

1 v-^ ~ -^ffp* VarS'p - 2 

— T /_, I P A n,kp- 

i=l 



Conditions (US]), (dZJ) and jTH]) imply 



Vn — > o~ 2 a.s. as n — > oo. 



This completes the proof of Proposition 13.151 □ 
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4 Proofs of the theorems 

4.1 Bootstrap for the sample mean under dependence conditions 

Proof. Proof of Theorem 12.11 Under condition 1 (strong mixing), this theorem can be 
found in the book of Lahiri [2003j. Theorems 3.1 and 3.2. For the second condition 
(near epoch dependence), we will first show that the central limit theorem holds. By 
our assumption, we have that 

2+8 

E\X -E[X \Z_ l ,...,Z l ]\T+-s 



E 



2+8 



\X -E[X \Z_ l ,...,Z l }\T+st 
+ E 



Xr, — E \Xn Z 



_ i 

{\Xa-E[X \Z- U ... > Z l \\<a l T + 3 '} 

2+8 



Z x pi 



i 

{\x -e\Xq\z_ w ..m\>*, "^1 



< a, 



(i+sy 



E | Xq — E \Xo | Z 



Z, 



6(2+6) 
. (1+6) 2 



+ a/ 1+ ^ E \X - E[X \Z_ h ...,Z t ]\ 2+S < Ca 



8(2+6) 
,U+8jZ 



So we have that 

oo 

(E \Xq — E[X \Z_i, ... ,Zi 



2+8 
1+8 



k=l 



1+8 OO g 

2+5 <C^op<oo, 
fc=i 



and by Lemma 13. II the partial sum is asymptotically normal. We write X n = X n (v) + 
X n (v) with 

X n {v) = E[X n \Z n — v , . . . , Z n + V ] 

and X n (y) = X n — X n (y). Note that the sequence (X n (v)) n£ ^ is strongly mixing with 
mixing coefficients ax{t) = ot{i — 2v). Then for the bootstrap version of the sample 
mean, we have that 



= ^Jkp (X* >kp (v) - X n>kp (v)) + ^fkp (X* kp (u) - X n , kp {vj) = I n + II n 

where I n is constructed from the sampe X\(u), . . . , X n {y) and II2 from the sample 
Xi(p), . . . ,X n {y). The first part of this theorem implies that for any fixed u € M, the 
bootstap is valid for X njkp (v), meaning that 

(25) 



sup 



P* y/kp(Kkp(") ~ X n,k P )(v) < X - <&( 



(26) 
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in probability as n — > oo with o (u) = lim n _> Var[-^ Y27=i ^i{ v )\- Now we will estimate 
the variance of II n 



Var 



™ = h E f (E - m)) 2 - £(E - w 

»=1 \ j e B* i 6 Bi 



+ -Var^) + . . . + X»] - pX^ p (*) 



where fi = fi(u) = EX\{v). Observe that E\Xi{v)\ 2+s < E\Xi\ 2+s < oo, so LemmaE 
implies that 

P (X n! k p (P) - fj) -)■ 
a.s. as n — > oo. For any e > 0, we have that by the Markov inequality 



< 



'( 


1 k j 




1 fe 

-V 

kp f-f 



(^)-m)) 5 



> e 



ep \ 



pVarXxiu) + 2 £(p - i)Cov(Xi(P),X,-(i/)) 
i=2 



< - ( (2iV + l)Var(X 1 (P))+2C7^ l\\X o -E[X o \Z_ l i j ,...,Z [ 02_ts+a^( J ~ 

j=N ^ 3 1+4 



where 1 < JV < p. Now the conditions of the theorem imply that we can choose a fixed 
JV£M and i/£t, such that 

A < e 



1 



sup||$(-^-)-$(-)|| <e 
Var[Xi(^) + ... + X p (^)] <e. 



We note that for this v the lines (|25|) and (|26|) hold, which together with these three 
inequalities imply the statement of the theorem. □ 

Proof of Theorem \2.SX The proof under the first assumption (strong mixing) is based on 
Proposition 13. 131 Lemma |3. II implies that conditions (|15|) and (|16|) of Theorem 13.131 are 
satisfied. It remains to prove lines (|18p and (|22p . W.l.g. we can assume that [i = and 
we will prove 



\ k ( v p 

E uE x {i-^)p+if - ^(E x 

1 i=i y j=i i=i 



a.s. as n — > oo. 
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By the Borel-Cantelli lemma, it suffices to show that for any e > 

oo 

E^ 



1=1 



max 

2<<n<2'+ 1 



1 k \ V 2 2 \ 



> e < 00. 



Taking into account that kp ^ n and lines ([3]), we have 



II := P \ max 

2'<n<2'+ 1 



< P I max 

n<2'+ 1 



< P I max 

m<k(2 l + 1 ) 



k{n) I p(2 l ) 

E ^E ^(i-l)p+i) 2 - 
i=l \ j=l 

fc(n) / p(2 ; ) 

E (E %-l)P+i) 2 ~ ES ; 
i=l \ j=l 

m / P(2 l ) 

E (E x (»-i)p+j) 2 - -^(20 



-.2 

3 P(2 ; ) 



> ek[2 l )p{2 ly 



> eC2 



> eC2 l 



From Lemma 13,41 we have for some s > 1 that 



p(2 ; ) 

(E x (*-i)p+j) 2 - ^ 



2 

p(2«) 



1/5 < C(S S p(2l) < C P {2 1 ) 



2s 



Now using Lemma 13.21 and taking into account line (|27p . we obtain 

From the condition ([3]), it follows that 

k l p 



y P I max 

f-' \ 2 ; <n<2 ; +! 



~ E ( E ^-i)p+i) 2 - es p 



kp 1 



i=i \ i=i 



> e < J^J; < OO. 



It remains to prove (|22p . i.e. 



Because of the Borel-Cantelli lemma, it suffices to show that 

00 / k 

E p 

1=1 



a.s. 



max 

2 l <n<2 l + 1 



En^E^^)^} 



> ek{2 l )p{2 1 ) ) < 
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Using Lemma |3, 51 we conclude 

/ 

III -P 



\ 



max 

2 l <n<2 l + 1 



k(n) 

E 

i=l 



V 



1 -H^x 



'p(2') 



1 P(2 ( ) 

-p( _: Vi^x) 



(i-l)v+3 



> ek(2 l )p(2 h 



<x 



( 



<p 



max 

m<fc(2 i + 1 ) 



E 

i=l 



( 



1 v-p(2 ! ) v 



/p(2') 



1 P(2 ! ) \ 
P( ; V Xi < X)\ 



> efc(2')p(2 h 



<- 



C(jfe(2 l + 1 ) + ib(2'+ 1 )nr i , ^a(*) 



i+l^fc(2 ; + 1 ) 



e 2 fc 2 (2V(20 
where a(i) = a ((t - l)p(2 l ) + l). As YXi " 00 < 

Ck(2 l+1 ) 



Hi < 



e 2 k 2 (2 l )p 2 (2 i y 



From condition ([3]) we get that 



Hi < oo, 



so we have completed the proof under strong mixing. For the second assumption (near 
epoch dependence), we will check the assumptions of Proposition 13.121 Lemma 13.6 
implies the conditions (|15p and (|16p . W.l.o.g. we will assume that EX\ = \i = 0. First 
we will prove line (|17p . Note that by stationarity and Lemma 13.81 we have for any a > 1 
and some C > 



E 



S P> 



< Cap. 



Theorem A of Serfling |1970j implies that 

2" 

< Cm(log 2m) 2 p. 
In order to prove line (|1T|) it suffices to show that for any e > 



E 





a 


2" 


max 

l<a<m 








i=l 





(28) 



E^< 



i=i 



max 

2 l <n<2 l + 1 



P 1/2 x n , kp 



> e < oo. 



(29) 
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Keep in mind that p = p(n) does not change for 2 l < n < 2 l+1 by condition @. By 
Chebyshev inequality and (|28p . it follows that 



P( max 

2 l <n<2 l + 1 



„l/2 y 



> e) < PI max 

\ 2 l <n<2 l + 1 



i=l 



> fc(2'y/ 2 (2')e 



< 



< 



k 2 (2 l )p(2 l )e 
C(log(2-fc(2 ; + 1 ))) 2 
k{2 l )e 



< 



Ck(2 l+1 ){log(2 ■ k{2 l+1 ))) 2 p(2 l ) 
k 2 (2 l )p(2 l )e 



The latter implies line f)29[) and hence (|17p is proved, so we can proceed verifying line 
(|18p . First we will prove the existence of the constant C > such that 



E 



< Cmp 2 



for m > 1. Stationarity and Lemmas 13.71 13.81 and 13.91 imply 



i=i 



4mE (S 2 - ES 12 ) 2 + 8 ( m ~ i + 1)^^ - ESl) ■ (S^i - ES 2 p ) 



< AE 



m—l 



Yl ( S P,i ~ ES i 



i=l 
i odd 



i=3 



m—l 



<Cmp 2 + CmY Y \ EX h X h X h X h- EX h X i2 EX h X h\ 



i=3 ii,i2&Bi 
3\,32&B 2 

m—l 



i=2 

OO 



<Cmp 2 + Cm J> 4 ( (a^])^ 5 + Wd^^ 1 }))^ 



<Cmp 2 + Cmp 2 Y J k2 ( («[|])^ + (/3([|]))^J <Cmp 2 



k=l 



Now again using Theorem A of Serfhng |197L)j . we obtain 



E 



max 

Ka<m 



1=1 



< Cm(log 2m) 2 p 2 . 



(30) 



If we can prove 



y P( max 

^ 2 l <n<2 l + 1 



i=i 



> ekp) < oo, 



(31) 
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line (|18p follows by the Borel-Cantelli lemma. Using Chebyshev inequality and (|30[) we 

get 



P max 

\ 2 l <n<2 l + 1 



i=l 



> ekp 



<P max 

\ l<m<fc(2'+ 1 ) 



i=l 



> ek(2 l )p{2 h 



< Ck(2 l+1 )(log(2k(2 l+1 ))) 2 -p 2 (2 l ) < C{log(2k(2 l+1 ))) 2 



ek 2 (2 l )p 2 (2 l ) 



ek(2 l ) 



The latter implies line (|31|) and hence (|18|) . It remains to check line (|19j) . In order to do 
that it suffices to show that for any e > and ei > 



1=1 



max 

2'<n<2'+ 1 



1 k 

i=l 



> ei < oo 



(32) 



where S P) i = YljeBi^j- Using Markov, Holder, Chebyshev inequalities, line (JH), and 
Lemma 13.71 we obtain 



P( max 

2 l <n<2 l + 1 



<P( max 

2 l <n<2 l + 1 



1 k 

i^ll S P^{\Sp,i\' 2 >^p} 

1=1 



i=i 



S Pji | 2 >efc(2 ! )p(2 ! )} 



> e 1 k(2 l )p(2 1 )) 



£'(max 2 ; <n < 2 i+i |I]Ll ,S 'p,i :1I {|S , p, I | 2 >efc(2 ; )p(2 i )} 



< 



< 



< 



e 1 fc(2')p(2' 



E?=l } £! ' 5 'p, t 1I {|S p , I | 2 >efc(2')p(2')} fc(2 ; + 1 ) J E5 ,2 l| (|5 . p| 2 >efc(2i)p(2i) | 



ei 



k{2 l )p(2 l ) 



ei 



k{2 l )p{2 1 



C{ESiYl 2 {P{\S p \ 2 > ek{2 l )p{2 l )f/ 2 C ES: 



eip(2') 



< 



ei -ek(2 l )p 2 (2 l ) 



: eieA;(2')p 2 (2 / ) ~ ei -e£:(2 z )' 
The latter implies line ([32]) and thus (fT9|) is proved. 



□ 



Proof of Theorem \2.3l The proof is based on Proposition 13.151 The verification of lines 
(fT5|) .(fT7" |) . (fT8|) can be done along the lines of the proof above and is hence omitted. 

□ 
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4.2 Bootstrap for U-Statistics 

Lemma 4.1. Let (X n ) nem be a sequence of r.v. 's and A C {1, . . . , n} 4 . Then there is a 
constant C, such that: 



EE* 



h< 2 ( x h , x t 2 ) h 2 ( x i 3 > x ti ) 

(ii,i 2 ,i3,i4)eA 



<C \E[h 2 (X h ,X i2 )h 2 (X i3 ,X i4 )]\, 

*1,*2,*3,*4 = 1 



where h 2 is defined by the Hoeff ding-decomposition. 
Proof. By triangle inequality: 



EE* 



E h *( x tvK)h2( x ! 3 ,K) 

(21,42,23,24)6 A 



< 



(pk) 2 (pk- l) 2 



(2'l,22,23,2 4 )eA 



The bootstrapped expectation of h 2 (X* ,X*) h 2 (X* 3 ,X* 4 ) (conditionally on (X n ) ngJN ) 
depends on the way the indices 11,1-2,13,14 are allocated to the different blocks. First 
consider indices ii,i 2 , 13,14 lying in four different blocks Bj 1 , Bj 2 , Bj 3 , Bj 4 (therefore, 
X* , . . . , X* are independent for fixed (X n ) ngM ). From the construction of the bootstrap 
sample for any four different blocks Bj 1 , Bj 2 , Bj 3 , Bj 4 : 

\E[E* [h 2 (Xt v Xt 2 )h 2 (X*,X*)]]| 

= \ E \-TA E ^ 2 ( X h+k lP , X i 2 +k 2 p) h 2 (X i3+k3 p, Xi 4+k4p ) ] J 



l<i\+k\p<n 
l<i 2 +k 2 p<n 
l<i3+kip<n 
I<i4+k4p<n 



— fc4 E \ E ^ 2 ( X il+kip, X i 2 +k 2 p) h 2 (X i3+k3p , X ii+ k 4 p)}\ 



l<il+fcip<n 
l<i 2 +k 2 p<n 
l<23+fe3P<ro 
I<i4+k4p<n 



\EE*[h{X h ,X i2 )h(X i3 

; X ii , 

22,23, 
xBj. 

I n 

— M E i X h ' X ^2 ) h i X h > X i4 



(21,12,13,24) 



11,12,13,24=1 
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As there are less than k 4 possibilities to choose these four blocks, one gets: 



/] | EE* [h (Xi L , Xi 2 ) h (Xi 3 , Xi 4 ] 



(il,»2,*3i»4)e-A 
4 diff. blocks 



< Y, \E[h(X h ,X h )h(X i3 ,X,, 

*l,*2,i3,i4=l 



'4 J 



As an example, let i\ and i 2 now lie in the same block with 12 — %% = d > 0, while 13, £4 
lie in two further blocks. X* and X* 2 are dependent, the value of X* is determined by 
the value of X* (conditionally on (X n ) ng]N ). To repair this, add up the expected values 
for all 12 such that 12 in the same block as i\ and take into account that there are at 
most k 3 possibilities for £1,13,14: 

\E[E* [h 2 [X* v Xl)h 2 (X* 3 ,Xi)]]\ 

~ X/ \E[h2{Xi 1 ,Xi 1+d )h2{Xi. il Xi i )}\ 



< 



k 3 



l<i\+k\p<n—d 
l<i 3 +k 3 p<n 
I<i4+k4p<n 



\E[E*[h 2 (Xl,Xt 2 )h 2 (Xt 3 ,Xl)]]\ 



«2 

(il,i2,23,«4)£^ 
22 i n same block as 

n 



il,*2,«3,*4 = l 



|£[E-[ft 2 (X-,A'-)A 2 (A'-,X*)]] 



(ii,^2,i3,u)GA 

22 i n same block as Zl 



«l,«2,i3,«4=l 



*4 J 



When the indices are allocated to the blocks in another way, analogous arguments can 
be used, which completes the proof. □ 



Proof of Theorem \2.4\ The proof under condition 1 (strong mixing) is similar to the 
proof of Theorem 2.2 of Dehling and Wendler [2010bJ and hence omitted. Under near 
epoch dependence, first note that by Lemma f3. Ill (h-j (Xn)) [lcB is near epoch dependent 



with approximations constants a' k < Ca^ +2S < Ck «(i+24) ; so ^fcLi a fc 1+l5 < 00 and 



by Theorem 12.1 



sup 



P* 



pk 



£>l {&))< 



i=l 



P 



9 

-4 Em* 



< x 
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in probability as n — > oo. Furthermore, by Lemma 13.101 and Lemma |4. H we have that 

E(^lU n (h 2 )) 2 ^0 

_ 1 n 

EE* (^h~U n (h 2 )) 2 <C— Yl \E[h 2 (X n ,X t2 )h 2 (X i3 ,X i4 )}\^0. 



il!*2,*3,*4 = l 

The Chebyshev inequality completes the proof. 



□ 



Proof of Theorem \2. 51 We first show that 



P 



pkU* (h 2 ) -> 



E 



P* 



pkU* (h 2 )->0 



1. 



With Fubinis theorem, we will then conclude that 



P 



pkU* (h 2 ) -»• 



1 a.s. 



(33) 



We set 



l<i\<i2<pk ' 



With the method of subsequences, it suffices to show that 

max \b n Q* — b 2 i-iQti-i \ — > a.s., 

2'- 1 <n<2 1 



(34) 
(35) 



as I — > oo. By the conditions 1. or 2. of the theorem and Lemma |3.10[ there exists a 
r\ > such that 

rt 

\E[h 2 (X h , X i2 ) h 2 (X i3 ,X iA )}\ = (n 3 ~") . (36) 

11,12,13,14=1 

We use Chebyshev inequalitity and Lemma 14. II to prove line (|34p . For every e > 0: 

oo 1 oo 

j>[Mft (h2)\ > e] < ^Y b l' EE * [Q*Q»)] 
1=1 1=1 

1 oo 2 l _. oo 

<C~Y b l E \E[h 2 (X il ,X i2 )h 2 (X i3 ,X i4 )]\ <C-Y^ vl <oo. 



i=l *l,«2,i3i*4=l 



Z=l 



line ()34p follows with the Borel-Cantelli lemma. To prove (|35|) . we first have to find a 
bound for the second moments, using a well-known chaining technique: 

max \b n Q* n - by-iQ^-A 

— ._ maX ;- d ^2'- 1 +i2 d - l( 52 i - 1 +i2 ti - 1 ~~ h l ~ 1 +(i~l)2 d ~ 1 Q2 l - 1 +(i-l)2 d - 1 

d=1 l-l,...,2 
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As for any random variables Yi,...,Y n : E (maxi = i n 

\Yi\Y < Zi=l EY ^ it follows 

that 



EE* 



max \b n Q*— boi-iQZi-i 

2'- 1 <n<2 1 



I 2 l 



d=l i=l 

/ 2 l ~ d 



b2 l - 1 +i2 d - 1 Q2 l ~ 1 +i2 d - 1 ~ ^2 i - 1 + (i-l)2 d - 1 Q2'- 1 + (i-l)2 d - 1 



</ 2h %-^+i2 d -^ EE * 



d=l i=l 



=1 8=1 



+ * E X/ 2 ( ^2 i - 1 +i2 d - 1 ~ ^2 i - 1 + ( l -l)2 d - 1 ) -E^* 



( 32^- 1 +(i-l)2' i - 1 



2 i-i + j 2 d 



^EE* 



d=i 



i=l 



^2l- 1 +i2«*~ 1 < 32 i - 1 + (i-l)2 d - 1 



i 2 



+ ' 2 (^2i-l+i2 d - 1 + & 2 i - 1 + (i-l)2 d " 1 J ^2i-l+i2 d - 1 ~ & 2 ; - 1 + (i-l)2 d " 1 

d=l i=l 



[Q2 2 - 1 



+ (i-l)2 d - 



<i 2 6^_! £ |£ [/la PQ 15 X i2 ) fr 2 (X i3 ,X i4 )]| < CZ 2 2^. 

*1,*2,*3,*4 = 1 

In the last line we used the fact that the sequence (b n ) n&w is decreasing, Lemma R~T1 and 
(1361) . It now follows for all e > 



z=i 



max |a n Qn — a 2 ;-iQ 2 ;-i| > e 

2 ( - 1 <n<2 i 



<^2-<<oc, 
1=1 



the Borel-Cantelli Lemma completes the proof of line ()35p . Furthermore, we have that 

(£[q;]) 2 <£[q; 2 ] 

and conclude that ^j^^~^ E [QV\ ~* a - s - We use now the Hoeffding-decomposition 



pk{T%{h)-E?> [U*(h)]) 
2 

+ 



pfc (pA; — 1) 



l<i<j'<pA; 



l<i<j'<pfc 
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By Proposition 2.11 and Lemma 2.15 of Borovkova et al. |2001j . we have that (hi(X n )) n( - K 

3+6 ' 



is near epoch dependent with approximation constants di < C ( ^fa\ + a* +& ) . So we can 
apply Theorem 12.21 to obtain 



sup 



P* 



pk 



i=l 



))<x 



2 

-= y; h x (Xi) < x 



a.s. 



and by Theorem 2.1 of Dehling, Wendler [2010b| 

\fnU n (/12) — > a.s. 

Since y/pkU* (hz) — > 0, ^/pkE*U* {J12) — > a.s. have been already proved, (fT4"|) follows 
with the Lemma of Slutzky. To prove f j 1 3 j) . first recall that by Theorem! 



Var* 



kp 
1=1 



Var 



i=l 



a.s., 



and by Lemma 13. 101 Var y/nU n (/12) — > 0. Similar to the proof of (|33p . one can show that 
Var*VpfeC/* (/12) a.s., so (fl~3j) follows, which completes the proof. □ 
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